How the brain decides what we see.

Until recently it was thought that vision shows us what is actually ‘out there’ at any moment. It was supposed that all the images printed on the retina are coded and transmitted to the cortex; thus every detail of our visual experience would mirror faithfully the ongoing events in the stimulus field. The first clues that things are not that simple came from clinical investigations by Ramachandran and others.1 These showed that visual scotomata are not experienced as blank areas in the visual field but that the brain fills them in with (hallucinatory) sensations that the brain computes would most probably have been in this locus if there had been no scotoma. Clearly the brain mechanisms responsible for this ‘filling in’ were not evolved merely to fill in scotomata. They must have some wider function. Next, experiments in visual psychology have revealed that the brain will often construct sensations that override the visual input if the brain judges that the latter is too improbable. A notable example is reported by Kovács et al.2 These workers took two photographs, one of a monkey’s face and the second of a leafy tropical jungle. They converted these into two pastiches each composed of portions of each photo so that in the location where one photo showed part of the monkey’s face the other showed leafy jungle. Then each pastiche was shown separately to each retina so that retinal rivalry occurred. Under these circumstances, the subject did not see what was actually there—i.e., the two pastiches alternating—but rather a complete monkey face alternating with a complete leafy jungle. Clearly the brain had suppressed the improbable mixed pastiche in favour of what it was familiar with (and thus computed what was more probable). It was then discovered that, during a saccade (rapid movement of the eye), information coming from the eye is suppressed and what we see is largely virtual reality created by the brain from memory.3 Kleiste et al.4 expressed it thus: